“Mixed reality” generally refers to the merging of real and virtual worlds to produce new environments and visualisations in which physical and digital objects co-exist and interact in real time. Interactive input systems that utilize mixed reality, often referred to as “augmented interactive input systems”, are known in the art. Some existing augmented interactive input systems make use of a document camera to track physical objects such as cubes and flashcards that comprise recognizable patterns. Digital content is superimposed on top of the images captured by the document camera. Direct association of the physical objects to digital content allows users to manipulate the digital content through manipulation of the physical objects.
Stereoscopic three-dimensional (3D) input systems have also been considered. For example, U.S. Patent Application Publication No. 2008/0094398 to Ng et al. discloses a 3D visualization system that can be ported to a laptop or desktop personal computer (PC), or other standard two-dimensional (2D) computing environment, which uses a mouse and keyboard as user input devices. A cursor or icon can be drawn at a contextually appropriate depth, thus preserving 3D interactivity and visualization while only having available 2D control. A spatially correct depth can be automatically found for, and assigned to, the cursors and icons associated with various 3D tools, control panels and other manipulations. This can preserve the 3D experience of interacting with a 3D data set even though the 2D interface used to select objects and manipulate them cannot directly provide a third dimensional co-ordinate. Based upon the assigned position of the cursor or icon in 3D, the functionality of a selected tool, and whether and in what sequence any buttons have been pressed on a 2D interface device, a variety of 3D virtual tools and functionalities can be implemented and controlled by a standard 2D computer interface.
U.S. Patent Application Publication No. 2006/0012675 to Alpasian et al. discloses an apparatus and method for 3D interaction with an autostereoscopic display. A motion tracking system may include video cameras that track a 3D motion of a user within an interaction volume defined by the fields-of-view of the video cameras, as the user moves a light source or other optical marker or an anatomical region of the user within the interaction volume. The motion tracking system may generate 3D tracking data containing position information about the 3D motion. An imaging system may create a virtual scene by tracing 3D virtual objects in virtual space, using the position information in the 3D tracking data. The imaging system may synthesize a plurality of views of the virtual scene, and interlace the plurality of views to generate an interlaced image to drive the autostereoscopic display and to be displayed thereon.
A disadvantage of known augmented interactive input systems is the temporary loss of physical objects that are being tracked. This typically occurs when a recognizable pattern on the object is partially or completely obscured, such as by fingers or hands during manipulation of the physical object in a 3D input space. This can also occur when the recognizable pattern is out of the field of view of the camera, due to a particular orientation of the physical object in relation to the field of view of the camera. When the augmented interactive input system temporarily loses track of the recognizable pattern and then regains tracking of the recognizable pattern, the associated digital content shown on the display disappears momentarily and then re-appears, causing the display to flicker.
Additionally, while some known augmented interactive input systems allow manipulation of recognized image objects from a printed medium, such as paper, these systems do not provide any means of visual feedback on the paper to facilitate manipulation of the recognized image objects.
Improvements are generally desired. It is therefore an object of the present invention at least to provide novel interactive input systems and novel computerized methods.